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Abstract — Face recognition is a biometric approach that employs automated methods to verify or recognize 
the identity of a living person based on his/her physiological characteristics. In general, a biometric 
identification system makes use of either physiological characteristics (such as a fingerprint, iris pattern, or 
face) or behavior patterns (such as hand-writing, voice, or key-stroke pattern) to identify a person. Because 
of human inherent protectiveness of his/her eyes, some people are reluctant to use eye identification systems. 
Face recognition has the benefit of being a passive, non-intrusive system to verify personal identity in a 
"natural" and friendly way. 



I. Introduction 

Face recognition is a biometric approach that employs automated methods to verify or recognize the identity 
of a living person based on his/her physiological characteristics. In general, a biometric identification system 
makes use of either physiological characteristics (such as a fingerprint, iris pattern, or face) or behavior patterns 
(such as hand-writing, voice, or key-stroke pattern) to identify a person. Because of human inherent 
protectiveness of his/her eyes, some people are reluctant to use eye identification systems. Face recognition has 
the benefit of being a passive, non-intrusive system to verify personal identity in a "natural" and friendly way 
[5]. 

There are numerous possible applications for facial image processing algorithms. The most important of them 
concern face recognition. In this regard, one has to differentiate between closed worlds and open world settings. 
In a closed world application, the algorithm is dedicated to a limited group of persons, e.g. to recognize the 
members of a family. In an open world context the algorithm should be able to deal with images from 
"unknown" persons, i.e. persons that have not been presented to the system during its design or training. For 
example, an application indexing large image databases like Google images or television programs should 
recognize learned persons and respond with "unknown" if the person is not in the database of registered persons 
[5] [6]. 

Concerning face recognition, there further exist two types of problems: face identification and face 
verification (or authentication). The first problem, face identification, is to determine the identity of a person on 
an image. 

The second one only deals with the question: "Is „X" the identity of the person shown on the image?" or "Is 
the person shown on the image the one he claims to be?". These questions only require "yes" or "no" as the 
answer. 

Face recognition system can help in many ways: 

1) Checking for criminal records. 

2) Enhancement of security by using surveillance cameras in conjunction with face recognition system. 

3) Finding lost children's by using the images received from the cameras fitted at some public places. 

4) Knowing in advance if some VIP is entering the hotel. 

5) Detection of a criminal at public place. 

6) Can be used in different areas of science for comparing an entity with a set of entities. 

© 2013, IJCSMC All Rights Reserved 28 



Prof Sapana A Fegade et al, InternationalJoumal of Computer Science and Mobile Computing Vol.2 Issue. 1, January- 2013, pg. 28-32 



The system that is designed for automatic analysis of facial actions is usually called Facial Expression 
Recognition System (FERS). The FER system is composed of 3 main elements: face detection, feature 
extraction and expression recognition. Section 2 reviews the various methods for face detection. Section 3 deals 
with various methods for extracting features from face images and section 4 deals with expression classification. 

II. Face Detection 

As it was mentioned before, FER system consists of 3 stages. In the first stage, system takes input image and 
performs some image processing techniques on it in order to find the face region. System can operate on static 
images, where this procedure is called face localization or videos where we are dealing with face tracking. 

Major problems which can be encountered at this stage are different scales and orientations of face. They are 
usually caused by subject movements or changes in distance from camera. Significant body movements can also 
cause drastic changes in position of face in consecutive frames what makes tracking harder. What is more, 
complexity of background and variety of lightning conditions can be also quite confusing in tracking. For 
instance, when there is more than one face in the image, system should be able to distinguish which one is being 
tracked. Last but not least, occlusions which usually appear in spontaneous reactions need to be handled as well. 

Problems mentioned above were a challenge to search for techniques which would solve them. Among the 
techniques for face detection, we can distinguish two groups: holistic where face is treated as a whole unit and 
analytic where co-occurrence of characteristic facial elements is studied. 

Holistic face models: 

• Huang and Huang [7] used Point Distribution Model (PDM) which represents mean geometry of human 
face. Firstly, canny edge detector is applied to find two symmetrical vertical edges which estimate the face 
position and then PDM is fitted. 

• Pantic and Rothkrantz [8] proposed system which process images of frontal and profile face view. Vertical 
and horizontal histogram analysis is used to find face boundaries. Then, face contour is obtained by 
Thresholding the image with HSV color space values. 

Analytic face models: 

• Kobayashi and Hara [9] used image captured in monochrome mode to find face brightness distribution. 
Position of face is estimated by iris localization. 

• Kimura and Yachida [10] technique processes input image with an integral projection algorithm to find 
position of eye and mouth corners by color and edge information. Face is represented with Potential Net 
model which is fitted by the position of eyes and mouth. 

All of the above mentioned systems were designed to process facial images, however, they are not able to 
detect whether the face is present in the image. Systems which handle arbitrary images are listed below: 

• Essa and Pentland [11] created the "face space" by performing Principal Component Analysis of Eigen 
faces from 128 face images. Face is detected in the image if its distance from the face space is acceptable. 

• Rowley et al. [12] proposed neural network based face detection. Input image is scanned with a window and 
neural network decides if particular window contains a face or not. 

• Viola and Jones [13] introduced very efficient algorithm for object detection with use of Haar-like features 
as object representation and Adaboost as machine learning method. This algorithm is widely used in face 
detection. 



III. Feature Extraction 

After the face has been located in the image or video frame, it can be analyzed in terms of facial action 
occurrence. There are two types of features that are usually used to describe facial expression: geometric 
features and appearance features. Geometric features measure the displacements of certain parts of the face such 
as brows or mouth corners, while appearance features describe the change in face texture when particular action 
is performed. Apart from feature type, FER systems can be divided by the input which could be static images or 
image sequences. 

The task of geometric feature measurement is usually connected with face region analysis, especially finding 
and tracking crucial points in the face region. Possible problems that arise in face decomposition task could be 
occlusions and occurrences of facial hair or glasses. Furthermore, defining the feature set is difficult, because 
features should be descriptive and possibly not correlated. 

Feature extraction methods: 

• Pantic and Rothkrantz [8] selected a set of facial points from frontal and profile face images. The expression 

is measured by a distance between position of those points in the initial image (neutral face) and peak image 

(affected face). 



© 2013, IJCSMC All Rights Reserved 29 



Prof Sapana A Fegade et al, InternationalJoumal of Computer Science and Mobile Computing Vol.2 Issue. 1, January- 2013, pg. 28-32 



• Essa and Pentland [11] proposed temporal approach to the problem of facial expression analysis. They used 
the multiscale coarse-to-fine Kalman filtering. The facial motion is represented by spatio-temporal energy 
templates. 

• Black and Yacoob [14] introduced local parametric models of image motion based on optical flow 
information. Models could describe horizontal and vertical translation, divergence and curl. 

• Edwards et al. [15] used Active Appearance Model which is statistical model of shape and gray scale 
information. Relationship between AAM displacement and the image difference is analyzed for expression 
detection. Proposed system operates on static images. 

• Cohn et al [16] developed geometric feature based system in which the optical flow algorithm is performed 
only in 13x13 pixel regions surrounding facial landmarks. 

• Zeng et al. [17] used data extracted by the 3D face tracker called Piecewise Bezier Volume Deformation 
Tracker. The system was designed to recognize spontaneous emotions so three-dimensional tracking was 
beneficial. 

• Littlewort et al. [18] proposed system which uses only appearance features to describe facial expressions. 
Facial texture is measured by Gabor waveletes. 

• Shan et al. [19] investigated the Local Binary Pattern method for texture encoding in facial expression 
description. Two methods of feature extraction were proposed. In the first one, features are extracted from 
fixed set of patches and in the second method from most probable patches found by boosting. 



IV. Expression Recognition 

The last part of the FER system is based on machine learning theory; precisely it is the classification task. 
The input to the classifier is a set of features which were retrieved from face region in the previous stage. The 
set of features is formed to describe the facial expression. Classification requires supervised training, so the 
training set should consist of labeled data. Once the classifier is trained, it can recognize input images by 
assigning them a particular class label. The most commonly used facial expressions classification is done both in 
terms of Action Units, proposed in Facial Action Coding System and in terms of universal emotions: joy, 
sadness, anger, surprise, disgust and fear. There are a lot of different machine learning techniques for 
classification task, namely: K-Nearest Neighbors, Artificial Neural Networks, Support Vector Machines, Hidden 
Markov Models, Expert Systems with rule based classifier, Bayesian Networks or Boosting Techniques 
(Adaboost, Gentleboost). 

Three principal issues in classification task are: choosing good feature set, efficient machine learning 
technique and diverse database for training. Feature set should be composed of features that are discriminative 
and characteristic for particular expression. Machine learning technique is chosen usually by the sort of a feature 
set. Finally, database used as a training set should be big enough and contain various data. Approaches 
described in the literature are presented by categories of classification output. 

Action Units classification: 

• Pantic and Rothkrantz [8] introduced the expert system with rule based classifier, which can recognize 3 1 
action units with accuracy rate of 89%. 

• Cohn et al. [16] performed recognition with use of discriminant functions. Proposed method can distinguish 
8 AUs and 7 AUs combinations. Tests were performed on 504 image sequences of 100 subjects and the 
system obtained accuracy rate of 88%. 

Emotions classification: 

• Huang and Huang [7] detected motion by analysis of difference image between neutral and expression 
image. The minimum distance classifier is used for recognition of six basic emotions. Recognition result is 
84.5% 

• Kobayasi and Hara [9] used 234x50x6 neural back propagation network for recognition of 6 basic emotions. 
The achieved recognition accuracy is 85%. 

Zeng et al [17] used Support Vector Data Description (SVDD) with Kernel Whitening to avoid influence of 
nonhomogeneous data distributions in input space. The accuracy of a system is approximately 83%. 

• Littlewort et al. [18] introduced method called AdaSVM where facial expression is represented by Gabor 
wavelet coefficients. Firstly, the Adaboost method is applied and the most probable features are chosen by the 
highest value of frequencies. Then, reduced expression representation is the input to SVM classifier. System 
obtains 97% accuracy of generalization to novel subjects. 

• Pantic and Rothkranz [8] in their Expert System implemented also the rule based classification of emotions 
with use of previously recognized action units. For example, happiness is a combination of AU6, AU12, 
AU16, AU25. Blended emotions are allowed. 
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The result can be: 75% of happiness if only AU6, AU12, AU16 occurred. Accuracy achieved by a system is 

91%. 

A large amount of effort has been focused on describing facial expression features. Based on the feature in 
use, as introduced earlier, we can broadly divide the methods into three categories, i.e., geometry-based 
approaches, appearance based approaches, and the combination of the two. Geometry based approaches track 
the facial geometry information based on a set of facial landmark points over time and classify expressions 
based on their deformation. On the other hand, appearance-based approaches use information from the facial 
texture described by various types of texture descriptors, such as LBP, Gabor wavelets, and LPQ. The dynamics 
of the texture deformation can also be included for feature extraction. 



V. Conclusion 

Face recognition system is a complex image-processing problem in real world applications with complex 
effects of illumination, occlusion, and imaging condition on the live images. It is a combination of face 
detection and recognition techniques in image analyzes. Detection application is used to find position of the 
faces in a given image. Recognition algorithm is used to classify given images with known structured properties, 
which are used commonly in most of the computer vision applications. The goal of this paper is to review the 
facial expression recognition system. On a basis of the extensive study of different approaches to the problem of 
face action representation, appropriate algorithm can be selected for each stage of a system. 
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